In a single-chip color camera, each pixel of a CCD is covered with color filter. Red, Green, and Blue are typical colors used for color filter. The color filters are arranged in a mosaic pattern, and only one primary color is captured for every pixel. The mosaic pattern is referred to as a CFA (“CFA” means Color Filter Array) pattern. Therefore, data obtained through a CFA is an image in color mosaic, and it is imperfect as a full color image.
In order to produce a full color image, missing color channels need to be estimated from raw data of color mosaic. Generally, this color reconstruction process is referred to as demosaicing. The simplest demosaicing method is linear interpolation applied to every color channel. More sophisticated demosaicing methods (see Reference Literatures [1] and [2]) have also been reported, which have the accuracy of image reconstruction higher than that of linear interpolation.
Generally, the major problem in demosaicing is false colors that occur in the resulting color image. Although an image is filtered through an optical low-pass filter to reduce false color, it results in causing a problem that band limitation is interposed. Particularly, when an image is magnified for display on a high resolution screen, a problem arises that the degradation of image quality caused by band limitation is noticeable.
The resolution of a color image reconstructed by the demosaicing method is equal to the physical resolution of a CCD. For example, in the case of display, printing, post-processing, etc., a much higher resolution is required. Interpolation is a traditional solving method for such demands.
However, when compared with an image obtained by a CCD having higher resolution, interpolation results in a low quality image. This is caused that the interpolation process does not substantially add information. Interpolation has a problem that it cannot reconstruct the detail of the image. In other words, interpolation cannot reconstruct high frequency signals.
Super-resolution (see Reference Literatures [3] to [5]) is a method different from interpolation, and super-resolution can detailedly reconstruct high frequencies contained in a captured scene. Super-resolution is an image processing technique that combines a high resolution image from multiple low resolution images. Among various methods for super-resolution in literatures, a frequency domain method that had proposed by Tsay and Huang first proposed a theory of super-resolution (see Reference Literature [3]). Peleg et al. proposed a spatial domain method based on a recursive back projection method (see Reference Literature [4]). First, image capturing process of a high resolution image is simulated to generate a measurement value, and simulation error is used for updating temporary image estimation error.
Up to now, super-resolution has been applied for gray scale images or full color images, but it has not been yet applied to raw data obtained by a single-chip CCD. Although demosaicing and super-resolution are sequentially executed to obtain a high resolution color image, such methods cause damage because of false color and blur effect seen in a demosaiced image. Since data available for a user is a demosaiced (and also compressed) image, such damage is often seen when a consumer camera is used.
The invention has been made in view of the circumstances described above. An object of the invention is to provide a high resolution color image reconstruction method, a high resolution color image reconstruction apparatus, and a high resolution color image reconstruction program, which can overcome the limitation described above, and directly use a raw color mosaic image captured by a CFA-masked CCD to reconstruct a high resolution color image from a single-chip CCD array.
The invention is characterized by a single step process for offering a spatial resolution higher than the physical resolution of a CCD. Accordingly, the invention allows an effective integration of demosaicing and an increase in resolution.